Skip to content

docs(dr): add CNPG StorageClass migration runbook (longhorn → longhorn-wffc) - #2424

Merged
devantler merged 2 commits into
mainfrom
claude/cnpg-storageclass-migration-runbook
Jul 3, 2026
Merged

devantler merged 2 commits into
mainfrom
claude/cnpg-storageclass-migration-runbook

Conversation

@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Daily AI Assistant

Why

Phase 2 of node right-sizing. Phase 1 (#2421) adds the longhorn-wffc WaitForFirstConsumer StorageClass; the four CNPG databases still sit on the old Immediate longhorn class, which is what strands their replicas on autoscaler overflow nodes (verified: umami-db-8 is on a cx33 autoscale node).

What

Adds a DR runbook documenting the no-data-loss procedure to migrate a CNPG cluster onto longhorn-wffc: recreate one replica at a time (CNPG re-clones each from the primary), verify health/replication between every step, and switch the primary over last. Ordered least→most critical (umami → backstage → coroot → wedding).

Operational note

Execution is gated on #2421 being promoted, merged, and deployed (the target StorageClass must exist first). I will run it cluster-by-cluster with verification gates once that lands — I do not self-promote #2421. Non-CNPG longhorn PVCs (kubescape/headlamp/actual-budget) are called out as backup-first, individual cases.

…n-wffc)

Phase 2 of node right-sizing: a no-data-loss procedure to move CNPG clusters
off the Immediate-binding longhorn StorageClass onto longhorn-wffc, by recreating
one replica at a time (re-cloned from the primary) and switching the primary over
last. Gated on platform#2421 (creates longhorn-wffc).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 3, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Adds a new runbook for migrating CloudNativePG clusters from Longhorn Immediate to WaitForFirstConsumer storage classes without data loss, including prerequisites, stepwise migration, verification, rollback, and operational notes.

Changes

Migration Runbook Documentation

Layer / File(s) Summary
Overview, rationale, and prerequisites
docs/dr/cnpg-storageclass-migration.md
Introduces the migration goal, explains the replica recreation flow, and lists the required prerequisites and execution order.
Pre-flight checks and manifest update
docs/dr/cnpg-storageclass-migration.md
Defines shared variables, Step 0 validation commands, and the StorageClass manifest update for new PVCs.
Replica and primary migration procedure
docs/dr/cnpg-storageclass-migration.md
Describes sequential replica PVC and pod recreation, then the primary switchover and recreation flow.
Post-checks, rollback, and operational notes
docs/dr/cnpg-storageclass-migration.md
Adds post-migration validation, rollback guidance, plugin-free execution notes, and constraints for non-CNPG PVC migration.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the new CNPG StorageClass migration runbook and names the source and target classes.
Description check ✅ Passed The description matches the changeset by explaining the CNPG migration runbook, prerequisites, and execution order.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/cnpg-storageclass-migration-runbook

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/dr/cnpg-storageclass-migration.md`:
- Around line 59-67: The replica cleanup commands are not copy-pasteable because
$<inst> is invalid shell syntax. Update the example in the migration doc to use
a real shell variable or a consistent literal placeholder, and make the commands
in the delete/get steps reference that same value via the existing inst
placeholder so readers can run the sequence directly.
- Around line 75-81: The switchover example in the CNPG migration guide still
uses a placeholder patch target, which is not a valid user-facing command.
Update the example in the documentation flow around the cnpg promote/switchover
step to use the actual supported `kubectl cnpg promote $CL <migrated-replica>`
command, and if needed mention the fallback operator-supported flow explicitly
rather than the placeholder JSON.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 156f2128-feb3-44b8-8ad0-bb04f1c2ca79

📥 Commits

Reviewing files that changed from the base of the PR and between c472544 and 2625db7.

📒 Files selected for processing (1)
  • docs/dr/cnpg-storageclass-migration.md
📜 Review details
🧰 Additional context used
📓 Path-based instructions (1)
docs/dr/**

📄 CodeRabbit inference engine (AGENTS.md)

Keep disaster-recovery runbooks and recovery procedures under docs/dr/.

Files:

  • docs/dr/cnpg-storageclass-migration.md

Comment thread docs/dr/cnpg-storageclass-migration.md Resolved
Comment thread docs/dr/cnpg-storageclass-migration.md Resolved
@github-project-automation github-project-automation Bot moved this from 🫴 Ready to 🏃🏻‍♂️ In Progress in 🌊 Project Board Jul 3, 2026
…r step

Addresses CodeRabbit: $<inst> is invalid shell — use a real `inst` variable with
quoted expansions; and the spec.switchover patch was not a real CNPG field —
replace with the cnpg-plugin promote (or plugin-free delete-primary-pod failover).
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Daily AI Assistant

Fixed in a400f7a. The replica block now sets a real inst=<replica-name> variable and uses quoted "$inst"/"$NS"/"$CL" throughout, so it is copy-pasteable. I also corrected the primary step — the spec.switchover patch was not a real CNPG field; it now uses kubectl cnpg promote (graceful) with a plugin-free delete primary pod failover alternative.

@devantler
devantler marked this pull request as ready for review July 3, 2026 20:44
@devantler
devantler added this pull request to the merge queue Jul 3, 2026
Merged via the queue into main with commit 48e8005 Jul 3, 2026
15 checks passed
@devantler
devantler deleted the claude/cnpg-storageclass-migration-runbook branch July 3, 2026 20:45
@github-project-automation github-project-automation Bot moved this from 🏃🏻‍♂️ In Progress to ✅ Done in 🌊 Project Board Jul 3, 2026
@botantler-1

botantler-1 Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 1.96.4 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

1 participant